Skip to content

[SEA-NodeJS] (2/3) SEA execution + result fetching#410

Open
msrathore-db wants to merge 1 commit into
msrathore/sea-connect-authfrom
msrathore/sea-execution-results
Open

[SEA-NodeJS] (2/3) SEA execution + result fetching#410
msrathore-db wants to merge 1 commit into
msrathore/sea-connect-authfrom
msrathore/sea-execution-results

Conversation

@msrathore-db
Copy link
Copy Markdown
Contributor

@msrathore-db msrathore-db commented Jun 1, 2026

Second of three stacked PRs (base: 1/3 connect + auth). Wires statement execution + result reads.

  • SeaSessionBackend.executeStatement — real impl; runs SQL via the napi Connection, returns a SeaOperationBackend.
  • SeaOperationBackend — fetch pipeline (Statement.fetchNextBatch → SeaResultsProvider → ArrowResultConverter → ResultSlicer) + cancel/close/finished via SeaOperationLifecycle.
  • SeaResultsProvider / SeaArrowIpc / SeaArrowIpcDurationFix — Arrow IPC decode for inline + CloudFetch (duration-fix rewrites Arrow Duration → Int64 for apache-arrow@13).
  • ArrowResultConverter — neutral { schema? } constructor so Thrift + SEA both construct it without an adapter.
  • flatbuffers pinned to 23.5.26 (matches apache-arrow@13's nested copy).

Tests: executeStatement/openSession forwarding, M0 datatype round-trip (primitives + ARRAY/MAP/STRUCT), multi-batch streaming, neutral-metadata contract.

Stack: 1/3 → 2/3#411

This pull request and its description were written by Isaac.

Second of three stacked PRs (base: [1/3] connect + auth). Wires the
statement-execution + result-read path:

- SeaSessionBackend.executeStatement: real implementation — runs SQL via the
  napi Connection and returns a SeaOperationBackend (replaces [1/3]'s stub).
- SeaOperationBackend: fetch pipeline (napi Statement.fetchNextBatch →
  SeaResultsProvider → ArrowResultConverter → ResultSlicer) plus operation
  cancel/close/finished via the SeaOperationLifecycle helpers.
- SeaResultsProvider / SeaArrowIpc / SeaArrowIpcDurationFix: Arrow IPC decode
  for inline + CloudFetch result batches (the duration-fix pre-processor
  rewrites Arrow Duration → Int64 so apache-arrow@13 can read it).
- ArrowResultConverter: constructor now takes the neutral { schema? } shape so
  both the Thrift and SEA backends construct it without an adapter.
- flatbuffers pinned to 23.5.26 to match apache-arrow@13's nested copy.

Tests: executeStatement + openSession forwarding, M0 datatype round-trip
through the shared converter (primitives + ARRAY/MAP/STRUCT), multi-batch
streaming, and the neutral-metadata converter contract. Full INTERVAL-type
value parity + exhaustive operation-lifecycle coverage land in [3/3].

Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant